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Background 



Computer-assisted instruction is becoming important in 
all levels of education. Since 1960 the senior author and 
Mr. Grosvenor Rust conducted eleven controlled experiments 
in branching pictorial programming simulating computer- assisted 
instruction.^ The subjects were college freshmen, juniors, and 
seniors. The results of these 11 studies are compared in the 9 
experiments reported here with parallel experiments on high 
school seniors. The purpose of the present series was to deter- 
mine if information and skills necessary to use a university 
library could be taught in high school by this method of branching 
pictorial programming. 

The four branching programs used in both series of experiments 
contain basic information on the card catalog, encyclopedias and 
dictionaries, periodicals, and indexes. Programmers and library 
consultants devised the program through the usual programming 
system of setting objectives, writing criterion frames and 
instruction frames, devising diagnostic questions, writing the 
wrong answer frames and remedial circuits associated with them, 
and providing by-passes. Special devices for particular experi- 
ments were inserted as needed. Programs went through the usual 
routine of multiple pretesting revision. The paper and pencil 

^Wendt, McCoy, & Rust, "A Study to Determine the Extent to 
Which Instruction to University Freshmen in the Use of the 
Library Can Be Turned Over to Teaching Machines." U.SOE Grant 
No. 7-11-076.00, NDEA Title VII. 

Wendt & Rust, "To Test Refinements in Intrinsic Programming 
in Pictorial, audio and Performance Frames to Maximize the 
Probability of Desired Terminal Behavior." U.SOE Grant No. 7-23- 
0907-189, NDEA Title VII. 
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criterion test was established by using professional librarians 
as subjects and consultation with other professional librarians. 

All instruction frames were combined kodachrome picture and 
print projected as slides on a screen in front of the subject and 
in some experiments the print content was eliminated and presented 
by audio tape to the subject while he was looking at the colored 
picture. Both the slide projectors and the audio tape player of 
course had to be random access as is the case with CAT. 

A special feature of the program.s were performance frames. 

At points during the program a frame called for the student to 
stop the progr2un and perform a skill simulating the terminal 
skill to be required of the subject. For example, after some 
instruction in the card catalog the subject was directed to find . 
a particular catalog card in the sample catalog tray before him. 

A diagnostic question then checked his answer. If wrong, he was 
directed to a wrong answer frame which explained what he had done 
wrong and returned him to the performance frame. Other simulation 
skills required of the subject were finding an item in a diction- 
ary or in an encyclopedia, locating a particular periodical or a 
book, or getting information out of an appropriate index. The 
performance frames were designed in accordance to the most basic 
principle in programming — to help the learner exhibit the speci- 
fied terminal behavior in the shortest time. 

To measure terminal behavior a performance rating scale 
was devised and stahdardized. This consisted of assignments 
given to each subject on the floor of the library; in carrying 
out these assignments actions of the subject were rated by an 




observer. 
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Brief Summary of Previous Results 

The pictorial frames and the performance frames were found 
to have no significant effect on the paper and pencil criterion 
test but highly a significant effect on the performance rating 
scale. 

Method 

Subjects used in the present experiment were seniors at 
the Carbondale Community High School and at the University 
Laboratory School. The original plan was to use only students 
from CCHS but signed permission was required from each parent 
and so many parents refused this permission that not enough 
subjects were available. It was necessary, therefore, to run 
additional experiments using the laboratory school. 

Each student sat before a console facing a screen in a 
small room. Behind him in a sound insulated room four Sarkes- 
Tarzian projectors were controlled by the key board in front 
of the subject so that any individual slides could be projected 
on the screen by pushing the appropriate buttons. Each frame 
on the screen gave the subject the number of the next frame to 
which he should proceed. In the diagnostic questions each 
alternative answer gave the subject a different frame number 
to which to proceed. The program and equipment simulates the 
IBM 1500 CAI System in that both audio and pictorial stimuli 
are presented in any random order necessary through the use 
of the random access projectors and tape recorder. A print-out 
kept a record of each student's route through the many branches, 
and a timer kept a record of the time spent on each decision 
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and total time. Each program was charted like system on paper 
and each subject’s progress could be graphically portrayed. 

For the performance rating scale tests on the floor of the 
university library all the high school students, both CCHS and 
Laboratory School, were brought o the university library. 

All students were given a pretest and a post test which 
consisted of two forms of the paper and pencil criterion test. 

Experiment 1. Comparison of Carbondale Community High School 

and College Picture and Print Group ■ 

In the branching program all instruction frames consisted 
of a kodachrome projected picture accompanied by a few words of 
print. In most cases the picture and the print were of equal 
importance to the instructional value of the frame. To test 
the unique contribution of the pictures, in one program the 
pictures were eliminated, their content being translated into 
print which was added to the print already in the frame. This 
version of the program was called Print-only. In the previous 
series of experiments one group of college students which viewed 
the Print-only prograun while another used the regular program 
which included kodachrome photographs. There was no significant 
difference between the groups on the paper-and-pencil criterion 
test. 

In the present experiment two similar groups of seniors 

from the Carbondale. Community High School were used. One 

% 

group (N=12) used the Print-only program while the other (N=15) 
used the Picture-print prograun. 
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Analysis lA Gain Scores 

An analysis of variance was used on these four groups on 
their gain scores, the groups consisting of the two CCHS groups 
and the previous two college groups. The only significant 
value of F was between groups, indicating that the high school 
students had lower gain scores. (See Table 1) 

Analysis IB Rating Scale Scores 

The same four groups were compared on their scores on 
the performance rating scale measuring their carrying out of 
actual assignments on the floor of the library. An analysis 
of variance showed that there was no significant difference 
between the group, between the methods, or in the interaction. 

Analysis 1C Program time 

The four groups were compared on the basis of the total 
time it took the students to complete the program. An analysis 
of variance again showed no significant difference between the 
groups, between the methods, or in the interaction. 

Experiment 1 - Conclusions 

This experiment showed no difference between the high school 
seniors and the college students in the effectiveness of the 
pictures in the instruction frames. The high school seniors, 
like the college students, did not show any difference between 
the print-only methods and the picture-print methods as evaluated 
by the pencil paper criterion test, they merely scored lower. In 
the performance rating scale the college students had previously 
shown a significant difference in favor of the picture group but 
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no difference was noted in the high school seniors. With the 
college students it was hypothicated that although the pictures 
added nothing to the ability of the student to make a verbal 
response to the paper and pencil test, it made an important 
difference in the much more important criterion of terminal 
behavior in the library. If anything, high school students 
should be helped more by pictorial simulation of the terminal 
behavior than older college students, but this was not the case. 
At this stage of this type of investigation no possible 
explanation can be suggested. 

In the matter of total program time required to complete 
the program the high school seniors showed the same lack of 
difference between the Print-only and the Picture-print group 
as the college students had previously shown. 

Experiment 2. Comparison of University School and College , 

Picture-print and Audio-only 

Analysis 2 Criterion Test 

In a previous experiment with college students the print 
content of the Picture-print instruction frames was transferred 
to a random-access tape player. Thus the experimental students 
saw a kodachrome picture on the screen at the same time that 
they heard on earphones the words previously appearing on the 
screen as print with the picture. This version was referred 
to as the Audio-only version. A group of 29 seniors from the 
University High School was exposed to program 1 on Cataloging 
in this version. A comparable group of 21 University High 
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School seniors were exposed to the same program in its original 
Picture-print version. 

Analysis 2A Gain Scores 

• An analysis of variance of these groups showed no 
significance between the levels of the methods and no 
significant difference in the interaction. 

Analysis 2B Rating Scale 

The analysis of variance by rating scale scores showed 
a significance in the interaction. A further analysis by 
t-tests showed that this significance arose from differences 
in methods, that is, the Picture-print did significantly better 
in the performance rating scales than the Audio-only group. 

Analysis 2C Program Time 

The ratio for the four groups again showed a significant 
difference in the interaction and this was traced by t-tests 
to the difference between the high school students and the 
college students. 

Analysis 2D By-passes Attempted 

The significant interaction by an F-ratio was followed 
by t-test which showed that the significant difference was 
caused by more by-passes attempted by the high sfchool students 
than by the college students. 

Analysis 2E By-pass Success/attempts 

Again the significant more successes in by-passes by the 
high school students than by the college students. 
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Experiment 2 - Conclusions 

Again there was no significant difference between the two 
groups as measured by the paper and pencil criterion test, but 
there were significant differences by any other measure — rating 
scale score, progr 2 un time, by-pa''ses * htempted, and by-passes 
successfully achieved. Apparently the imparting of informatin 
by two sensory modes simultaneously, visual auditory, was not 
superior to the pure visual approach. Possibly the important 
factor is that the listener must receive an audio message at 
the given speed of the message but a reader can read as fast 
as he desires. 

Experiment 3. Comparison of combined high school classes 

against the college group on by-passes 
attempted and by-passes success 

A group of 28 college juniors and seniors in 1965 provided 

data for the number of by-passes attempted and the number of 
« 

successes after such attempts. To compare with their record 
the data of 15 seniors from the Carbondale Community High 
School and from 28 seniors in the University School were 
combined on the S 2 une two records of performance. 

Analysis 3A By-Passes Attempted 

T-tests comparing the high school seniors and the college 
students produced a value t=2.2271 which was significant to 5% 
level of confidence. The college students attempted many more 
by-passes than the high school students. 
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Analysis 3B The Ratio of By-pass Successes to By-pass Attempts 

On this measure there was no significant difference between 
the high school and the college group. 

Experiment 3 - Conclusions 

Apparently the high school students even though they 
attempted many less by-passes were just as successful in those 
by-passes which they did attempt. The ratio of by-pass 
successes to by-passes attempts was originally devised in 
order to get at some of the factors that induce subjects to 
try by-passes. Anormalities had previously been discovered in 
this matter. One was that there was no relation between by- 
passes attempted by college students and their pre-test , or to 
their grade point average or general ability. Similarly, many 
students who attempted a by-pass and after answering the entry 
question had earned the right to by-pass, requested to be put 
back on the main line, thereby throwing away the privi ledge of 
the by-pass which they had earned. All these anormalities indi- 
cated some factors which were unknown. One experiment employed 
Rotter's Incomplete Sentence blank in an attempt to Isolate 
some factors which might operating, but only two were slightly 
effective. The basic factors involving the decision to try 
by-passes is therefore still unknown. 

Experiment 4. Comparison of University School and College 

Audio- only 




As explained previously. Program 1 was transformed into 
an audio program by transferring the print appearing on the 
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screen to a random access tape player. The subject therefore 
saw the kodachrome picture on the screen and heard a voice 
giving the material that was formerly print. 

Analysis 4A Gain Scores 

The t-test revealed a value of 1.1375 which was not 
significant. 

Analysis 4B Progrcun Time 

The t value was 3.9250 which was significant at the 1% 
level of confidence in favor of the college group. 

Experiment 4 - Conclusions 

From the results it appears that high school group 
attained an equally high level of competence on the paper 
and pencil criterion test as the college students, but took 
significantly longer to do so. This is in conformance with 
the general results obtained from prograunmed instruction, after 
programmed instruction students tend to have smaller variauice 
in criterion scores because one of the great advantages of 
programmed instruction is that each student no matter what his 
can proceed at his own rate of speed. Apparently the 
fact that the audio portion of the program was paced had little 
effect on keeping the program times more nearly uniform. Although 
the audio components may have made the time spent on instruction 
frames more uniform greater variability was encountered on the 
diagnostic question frames and the resulting wrong answer circuits . 
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Experiment 5. Carbondale Community High School and College ^ 

Picture-print y grade point 

A group of seniors from CCHS (n=15) were compared with the 
college group in 1964 (n=18) both groups seeing the standard 
picture-print version , but each group dicotomized at a grade 
point average of 3.51 and above or 3.50 and below. 

Analysis 5 Pre-test Scores 

Analysis of variance showed no significant difference 
either between groups, between levels, or in interaction. 

Analysis 5A Gain Scores 

An analysis of variance indicated that an F of 4.57 was 
significant at the 5% level of confidence. There was no 
significant difference between the two groups nor in the 
interaction. A subsequent t-test by levels within each group 
showed no significant differences. 

Analysis 5B Rating Scale 

An analysis of variance by levels again showed a significant 
difference between the groups but not between the levels, nor in 
interaction. Subsequent t-tests showed a difference between the 
groups at the 5% level of confidence. 

Analysis 5C Program Time 

An Analysis of variance showed that there was no significant 
difference between the groups, between the levels, nor in inter- 



action 
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Analysis 5D Rating Scale Time 

This term means the time required by the subjects to complete 
the performance tests. An analysis of variance showed no signifi- 
cant difference either between groups, or between levels, or in 
interaction. 



Experiment 5 - Conclusions 

Of all the tests of this experiment only two were found 
to be significant, they were at the 5% level of confidence and 
they are difficult to interpret. For example, the significant 
between levels on the gain scores probably has little relation 
to the pictorial prograunti involved and is probably entirely 
dependent upon the ability of the students in both groups. 

The other significant difference was on rating scale scores 2nd 
was in favor of the college group. This means that the college 
students performed better on the floor of the library after the 
program than the high school seniors did. One might conclude 
that the pictorial content of the program had less effect than 
providing the high school seniors with a simulation of the terminal 
behavior that was to be required of them. 



Experiment 6 . A comparison of Picture-print versus Picture 

audio high school groups by GPA levels . 




One group of University High School seniors (N=45) was 
exposed to the usual Picture-print version of the program 
another group (N=18) was exposed to the Picture-audio program. 
Both were versions of Progr£un 1 on cataloging. Both groups 
were divided at a grade point average of 3.6 and below or a 
grade point average of 4.0 and above. 
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Analysis 6A Gain Scores 

An analysis of variance showed no significant difference 
either between groups, between levels, or in interaction. 

Analysis 6B Rating Scale Scores 

Analysis of variance gave a value of 35.18 between groups 
in favor of the audio group. This was significant difference 
between levels or in interaction. 

Analysis 6C Program Time 

Here the analysis of variance showed a significant 
difference at the 5% level between groups, at the 1% level 
between levels, and at the 1% level in interaction. Subsequent 
t-test showed that the upper ability students profited more from 
the regular picture-print version and the lower ad^ility students 
profited more from the picture-audio version. 

Analysis 6D Rating Scale Time 

Analysis of variance showed no significant difference 
between levels or in interaction but a significance difference 
at the 1% level between groups. 

Experiment 6 - Conclusions 

These varied results are very difficult to interpret in a 
systematic manner. As before gain scores on the paper and 
pencil criterion test apparently were a poor measure of the 
effectiveness of any particular version of the program of the 
ability of the students. In rating scale scores the audio group 
did significantly better. In program time the picture group 
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seemed to be superior to the audio group and at the same time 
students that had a higher grade point average were better than 
those with a lower GPA. Finally, the Picture-print seemed to 
help Lhe superior students more and the Picture-audio version 
seemed to help the less able students. Several factors seem 
to be at work: the paper and pencil test is a poor criterion 

the superior students, being good readers would naturally profit 
more from the Picture-print version than the Picture-audio 
version whereas the less able students would profit more from 
the slow pacing of the picture-audio version. No conclusions 
can be drawn at all concerning the value of one sensory mode 
of instruction versus two. 
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Table 1 



Experiment 1-A 
Gain Scores 

ANALYSIS OF VARIANCE TABLE (UNEQUAL N) 



Source 


DF 


SS 


MS 


F 


TOTAL 


77 


3326.15 


43.19 


1.08 


A Between 


groups 


1 


237.76 


237.76 


5.96* 


B Between 


method 


1 


78.08 


78.08 


1.95 


AB Interaction 


1 


60.09 


60.09 


1.50 


ERROR 


74 


2950.20 


39.86 


1.50 



*Significant at .05 
Table 2 

Experiment 1-B 
Rating Scale Scores 
ANALYSIS OF VARIANCE TABLE (UNEQUAL N) 



Source 


DF 


SS 


MS 


F 


TOTAL 


77 


22751.53 


295.47 


1.00 


A 


1 


220.87 


220.87 


.75 


B 


1 


4.73 


4.73 


.01 


AB 


1 


870.20 


870.20 


2.97 


ERROR 


74 


21655.72 


292.64 





Table 3 



Experiment 1-C 
Program Time 

ANALYSIS OF VARIANCE TABLE (UNEQUAL N) 



Source 


DF 


SS • 


MS 


F 


TOTAL 


77 


40801.03 


529.88 


1.02 


A 


1 


770.00 


770.00 


1.48 


B 


1 


698.26 


698.26 


1.34 


AB 


1 


975.56 


975.56 


1.88 


ERROR 


74 


38357.21 


518.34 











Table 4 












Experiment 2 








ANALYSIS 


Mean-X 


Mean-Y 


SD-X 


SD-Y 


t 


2A 


Gain Scores 


10.190 


7.931 


5.464 


5.161 


1.46030 


2B 


Rating Scale 


67.428 


73.413 


10.274 


9.994 


-2.02376* 


2C 


Program 

Time 


86.666 


66.655 


11.572 


15.154 


4.97146** 


2D 


By-passes 

attempted 


5.666 


7.586 


2.415 


1.637 


-3.28015** 


2E 


By-pass 

success/attempts .317 


.497 


.232 


.196 


-2.89840** 






♦Significant at .05 



Significant at .01 






Table 5 



Experiment 3 



ANALYSIS 


Mean-X 


Mean-Y 


SD-X 


SD-Y 


3A By-passes 
attempted 


7.886 


10.921 


1.528 


8.774 


3B By-pass 

success/attempted . 532 


.577 


.204 


.369 








*Significant 






Table 
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Experiment 4 




ANALYSIS 


Mean-X 


Mean-Y 


SD-X 


SDrY 


4A Gain scores 


3338.696 


2110.190 


112.607 


5.464 


4B Program 
Time 


62.939 


82.619 


13.970 


22.184 



-.68657 



t 

1.13754 

-3.92502** 
**Signif icant at .01 



Table 7 
Experiment 5 
Pre-test 





ANALYSIS OF 


VARIANCE TABLE 


(UNEQUAL N) 




Source 


DF 


SS 


MS 


F 


TOTAL 


32 


723.11 


22.59 


.93 


A 


1 


.17 


.17 


0.00 


B 


1 


15.15 


15.15 


.62 


AB 


1 


7.06 


7.06 


.29 


ERROR 


29 


700.72 


24.16 
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Table 8 



Experiment 5-A 
Gain Scores 
ANALYSIS OF VARIANCE TABLE (UNEQUAL N) 



Source 


DF 


SS 


MS 


F 


TOTAL 


32 


1420.24 


44.38 


1.21 


A 


1 


128.65 


128.65 


3.52 


B 


1 


166.99 


166.99 


4.57* 


AB 


]. 


66.29 


66.29 


1.81 


ERROR 


29 


1058.29 


36.49 










* Significant at .05 




Mean-X 


Mean-Y 


SD-X 


SD-Y t 




34.666 


30.722 


7.345 


5.623 1.69193 



Table 9 



Experiment 5-B 



Rating Scale 



ANALYSIS OF VARIANCE TABLE (UNEQUAL N) 



Source DF 

TOTAL 32 

A 1 

B 1 

AB 1 

ERROR 29 



Mean-X 

60.739 



SS 


MS 


F 




10306.54 


322.07 


1.16 




1684.80 


1684.80 


6.07* 




575.60 


575.60 


2.07 




• 

00 


1.81 


0.00 




8044.32 


277.39 

*1 


Significant 


at . 05 


Mean-Y 


SD-X 


SD-Y 


t 


75.700 


19.065 


9.129 


-2.29354 



1 

i 

5 



j 




Table 10 



Experiment 5-C 
Program Time 

ANALYSIS OF VARIANCE TABLE (UNEQUAL N) 



Source 


DF 


SS 


MS 


F 


TOTAL 


32 


25552.96 


798.58 


1.01 


A 


1 


991.37 


991.37 


1.26 


B 


1 


1614.06 


1614.06 


2.05 


AB 


1 


130.73 


130.73 


.16 


ERROR 


29 . 


22816.80 


786.78 





Table 11 
Experiment 5-D 
Rating Scale Time 

ANALYSIS OF VARIANCE TABLE (UNEQUAL N) 



Source 


DF 


SS 


MS 


F 


TOTAL 


32 


4472.96 


139.78 


1.04 


A 


1 


380.06 


380.06 


2.84 


B 


1 


140.16 


140.16 


1.05 


AB 


1 


84.74 


84.74 


.63 


ERROR 


29 


3867.99 


33.37 
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Table 12 



Experiment 6-A 
Gain Scores 





ANALYSIS OF 


VARIANCE TABLE 


(UNEQUAL N) 




Source 


DF 


SS 


MS 


F 


TOTAL 


42 


1340.79 


31.92 


.97 


A 


1 


7 . 19. 


7.19 


.22 


B 


1 


56.31 


56.31 


1.72 


AB 


1 


3.86 


3.86 


.11 


ERROR 


39 


1273.41 


32.65 








Table 13 










Experiment 6-B 








Rating Scale Scores 






ANALYSIS OF 


VARIANCE TABLE 


(UNEQUAL N) 




Source 


DF 


SS 


MS 


F 


TOTAL 


42 


4283.16 


101.98 


1.81 


A 


1 


1975.41 


1975.41 


35.18** 


B 


1 


85.18 


85.18 


1.51 


AB 


1 


32.75 


32.75 


.58 



** Significant at .01 



Table 14 



Experiment 6~C 
Program Time 





ANALYSIS OF 


VARIANCE TABLE 


(UNEQUAL N) 




Source 


DF 


SS 


MS 


F 


TOTAL 


42 


11262.79 


268.16 


1.77 


A 


1 


892.61 


892.61 


5.91* 


B 


1 


2720.61 


2720.61 


18.04** 


AB 


1 


1769.09 


1769.09 


11.73** 


ERROR 


39 


5880.47 


150.78 






* Significant at . 


05 ** Significant 



Table 15 
Experiment 6-D 
Rating Scale Time 

ANALYSIS OF VARIANCE TABLE (UNEQUAL N) 



Source 


DF 


SS 


MS 


F 




TOTAL 


42 


2190.00 


52.14 


1.27 




A 


1 


569.70 


569.70 


13.89** 




B 


1 


9.51 


9.51 


.23 




AB 


1 


11.32 


11.32 


.27 




ERROR 


39 


1599.44 


41.01 












** Significant at 


.01 




Mean-X 


Mean-Y 


SD-X 


SD-Y 


t 


Group 


81.500 


69.217 


13.132 


17.018 


2.55817 


Method 


67.600 


85.111 


15.553 


11.498 


-3.95109 



